The best way to use Yocto and Docker

Introduction

Yocto is a project which allow you to create your own embedded Linux distro. Typically you can create a distro for your Raspberry Pi, even for your Arduino UNO (... Yep Zephyr [zephyr] did it !).

../_images/zephyr_logo.png

Why are we going to speak about Docker ? First of all, I like to always have the latest version of the distro I'm using (now, a Fedora 30) and guess what, Yocto support the precedent version (28 and 29). The second point is that I used to open a VM supported and launch the compilation but you may know that using a VM is synonym of loosing a bunch of performances.

With Docker we're speaking about containerization, so performances won't be lost since it will run directly on our host machine (just another namespace and a chroot !).

../_images/doc_docker.jpg

A dedicated image pulled from the internet

It's important to NOT download any docker image over the Internet as well as you do not dowload dpkg or rpm in an other way than using your package manager !

Second rule, it's not because you found it on the Docker Hub that it's trustworthy !

Well, once it said, in who can we trust ?! The CROPS community[CROPS], great scott !

CROPS

CROPS stands for CROss PlatformS, and claims to be an cross platform development framework that leverages Docker Containers.

The main idea behind this framework is that it will able Windows users, Mac users and people like me using a version of a Linux distro not yet supported.

Behind this framework there is 3 major components :

  • CEED
  • TURFF
  • CODI

But those 3 components are a bit out of topic for the instant, let's focus on crops/poky.

You'll find the Github page here : https://github.com/crops

and the Docker Hub here : https://hub.docker.com/u/crops

Poky Container

GitHub : https://github.com/crops/poky-container

First if you haven't Docker installed yet, you can go there : https://docs.docker.com/install/

Then we'll pull the crops/poky image from the Docker Hub :

docker pull crops/pocky

If you want to check if the image is correctly downloaded :

[pfontaine@precision poky]$ docker image ls
REPOSITORY                            TAG                 IMAGE ID            CREATED             SIZE
crops/poky                            latest              0bfb6b4c24ff        7 days ago          746MB

Now you may already have a Yocto project on your laptop, in this case move in the folder associated with your terminal, else juste create a project folder anywhere you want.

For this tutorial I will take the second option

[pfontaine@precision Bureau]$ mkdir yocto_crops
[pfontaine@precision Bureau]$ cd yocto_crops/

Now that you have an empty directory, even if you could clone directly the Yocto repo from Git we will start launching now our Docker container based on crops/poky.

[pfontaine@precision yocto_crops]$ docker run --rm -it -v $(pwd):$(pwd) crops/poky --workdir=$(pwd)
pokyuser@5aa54ef11b4f:/home/pfontaine/Bureau/yocto_crops$ 

For those who only begin with docker let's describe each arguments :

docker run \ #run a process in a container
--rm \ # clean up the container and remove the file system when the container exists, it allow
# us to use the container as a Kleenex
-it \ # Attach the STDIN and allocate a pseudo TTY
-v  $(pwd):$(pwd) \ # Attach a volume, allow to share a directory with the host
# since you moved in the project directory you can use $(pwd), on the left of ':'
# it's host and on the right it's the container representation
# ie : your project is localised at /Desktop/myFancyProject/, it will 
# map the folder in you container at /Desktop/myFancyProject
crops/poky \ # The image we want to use for containerization
-- workdir=$(pwd) # Tell docker where you want to pop when the TTY allocated will pop

../_images/simple.jpg

Now you can clone the poky distro and specify the branch you want to use for your embedded project !

pokyuser@5aa54ef11b4f:/home/pfontaine/Bureau/yocto_crops$ git clone git://git.yoctoproject.org/poky
Cloning into 'poky'...
remote: Counting objects: 450968, done.
remote: Compressing objects: 100% (106693/106693), done.
remote: Total 450968 (delta 337418), reused 450575 (delta 337031)
Receiving objects: 100% (450968/450968), 157.83 MiB | 3.02 MiB/s, done.
Resolving deltas: 100% (337418/337418), done.
Checking connectivity... done.

pokyuser@5aa54ef11b4f:/home/pfontaine/Bureau/yocto_crops$ cd poky/
pokyuser@5aa54ef11b4f:/home/pfontaine/Bureau/yocto_crops/poky$ ls
bitbake  documentation  LICENSE.GPL-2.0-only  meta       meta-selftest  meta-yocto-bsp     README.hardware  README.poky  scripts
build    LICENSE        LICENSE.MIT           meta-poky  meta-skeleton  oe-init-build-env  README.OE-Core   README.qemu

To be sure it works I launched a compilation using the initial layers.

pokyuser@5aa54ef11b4f:/home/pfontaine/Bureau/yocto_crops/poky$ source oe-init-build-env 

### Shell environment set up for builds. ###

You can now run 'bitbake <target>'

Common targets are:
    core-image-minimal
    core-image-sato
    meta-toolchain
    meta-ide-support

You can also run generated qemu images with a command like 'runqemu qemux86'

Other commonly useful commands are:
 - 'devtool' and 'recipetool' handle common recipe tasks
 - 'bitbake-layers' handles common layer tasks
 - 'oe-pkgdata-util' handles common target package tasks

It will set environment variables in order to use bitbake, and move you in the build directory !

In order to accelerate the process and since I have a machine with a nice i7 processor I use to change the /build/conf/local.conf file with :

BB_NUMBER_THREADS = "8"
# Also, make can be passed flags so it run parallel threads e.g.:
PARALLEL_MAKE = "-j 8"

To do that you can use your text editor based on the host !

../_images/vim_mf.jpg

Then, here is the last step before having your own Linux distro !

pokyuser@5aa54ef11b4f:/home/pfontaine/Bureau/yocto_crops/poky/build$ bitbake core-image-minimal
Loading cache: 100% |###############################################################################################################################################################################| Time: 0:00:00
Loaded 1310 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.43.1"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "i686-poky-linux"
MACHINE              = "qemux86"
DISTRO               = "poky"
DISTRO_VERSION       = "2.7+snapshot-20190829"
TUNE_FEATURES        = "m32 core2"
TARGET_FPU           = ""
meta                 
meta-poky            
meta-yocto-bsp       = "master:a91448bf731ef29a841a93e991e7b75efb5f1e28"

Initialising tasks: 100% |##########################################################################################################################################################################| Time: 0:00:01
...

Depending on the performance of your machine it will last from several minutes to ... several hours !

But once it will be done you will have the power to emulate your distro in Qemu !

Just go back on the host, move into the poky folder and source the oe-init-build-env script.

finaly type :

[pfontaine@precision build]$ runqemu qemux86
runqemu - INFO - Running MACHINE=qemux86 bitbake -e...
runqemu - INFO - Continuing with the following parameters:

KERNEL: [/home/pfontaine/Bureau/yocto_crops/poky/build/tmp/deploy/images/qemux86/bzImage--5.0.19+git0+7f6e97c357_55dd15336b-r0-qemux86-20190829174628.bin]
MACHINE: [qemux86]
FSTYPE: [ext4]
ROOTFS: [/home/pfontaine/Bureau/yocto_crops/poky/build/tmp/deploy/images/qemux86/core-image-minimal-qemux86-20190829174628.rootfs.ext4]
CONFFILE: [/home/pfontaine/Bureau/yocto_crops/poky/build/tmp/deploy/images/qemux86/core-image-minimal-qemux86-20190829174628.qemuboot.conf]

runqemu - INFO - Setting up tap interface under sudo
...

../_images/qemu_done.png

Conclusion

Now you have the ability to compile a distro on whatever pc you want !

Thanks for reading :)